[ROCm] Remove BF16 workarounds for PaddleOCR-VL on HIP#5096
Open
austin1997 wants to merge 1 commit intoPaddlePaddle:developfrom
Open
[ROCm] Remove BF16 workarounds for PaddleOCR-VL on HIP#5096austin1997 wants to merge 1 commit intoPaddlePaddle:developfrom
austin1997 wants to merge 1 commit intoPaddlePaddle:developfrom
Conversation
The PaddleOCR-VL pipeline previously needed two ROCm-specific escape hatches:
* `_keep_in_fp32_modules = ["visual", "mlp_AR"]` on
PaddleOCRVLForConditionalGeneration kept the SigLIP vision tower and the
multimodal projector in FP32 because BF16 layer_norm and BF16 softmax were
not registered for HIP, so running the vision encoder in BF16 crashed.
* Four `paddle.is_compiled_with_rocm()` blocks in
`paddlex/inference/models/runners/paddle_static/runner.py` (lines 406-408,
462-464, 496-498, 505-507) called
`delete_pass("conv2d_add_act_fuse_pass")` and
`delete_pass("conv2d_add_fuse_pass")` because both PIR passes rewrite
conv2d+add[+act] into the `fused_conv2d_add_act` op, which only has a
cuDNN GPUDNN kernel — kernel dispatch then failed on ROCm.
These are addressed at the framework level by the upstream Paddle BF16 fix
(layer_norm + softmax registration on HIP, plus gating both PIR passes on
PADDLE_WITH_CUDA so they no longer run on HIP builds). With that wheel
installed, both PaddleX workarounds become unnecessary:
* Drop `_keep_in_fp32_modules` so the vision encoder + multimodal projector
run natively in BF16 on ROCm. End-to-end output matches the FP32-fallback
path on PaddleOCR-VL-1.5 (validated on MI300X / gfx942 / ROCm 7.2). This
overlaps with PaddlePaddle#5077; if PaddlePaddle#5077 lands first, the conflict is trivial.
* Drop all four `delete_pass` blocks under `paddle.is_compiled_with_rocm()`.
Once the framework PR lands, the two passes are no longer registered on
HIP wheels, so `delete_pass` becomes a no-op there.
Requires the framework BF16 PR to be merged and released; with older Paddle
wheels the BF16 visual path will still crash on ROCm. CUDA behavior is
unchanged — both passes remain registered under PADDLE_WITH_CUDA, and the
vision encoder simply uses whatever dtype the model is loaded with.
|
|
|
Thanks for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PaddlePaddle/Paddle#78711 在框架层适配了 HIP BF16 剩余的算子缺口(
layer_norm/softmax注册 BF16 内核,conv2d_add[_act]_fuse_pass在 HIP wheel 上不再注册)。配合已合入的 PaddlePaddle/Paddle#78587(BF16 conv 内核),PaddleOCR-VL 端到端 BF16 推理在 AMD GPU 上已经可以跑通。本 PR 同步移除 PaddleX 中针对 ROCm 的两类 BF16 workaround。修复 #5095。依赖 PaddlePaddle/Paddle#78711 与 PaddlePaddle/Paddle#78587 都合入并发版后才能合并——否则旧 wheel 上 BF16 视觉子图仍会崩溃。
Changes
paddlex/inference/models/doc_vlm/modeling/paddleocr_vl/_paddleocr_vl.py:删除_keep_in_fp32_modules = ["visual", "mlp_AR"],让 SigLIP 视觉塔 +mlp_ARprojector 跟随模型 dtype,无需强制 FP32。paddlex/inference/models/runners/paddle_static/runner.py:删除 4 处if paddle.is_compiled_with_rocm(): config.delete_pass("conv2d_add_act_fuse_pass"); config.delete_pass("conv2d_add_fuse_pass")块(行 406-408、462-464、496-498、505-507)。CUDA 推理行为完全不变。
与现有 PR 的关系
_keep_in_fp32_modules = None部分与 fix(doc_vlm): remove ROCm BF16 _keep_in_fp32_modules workaround in PaddleOCR-VL #5077(fchange,Hackathon)改动重叠。本 PR 同时把runner.py中 4 处delete_passworkaround 一并清理(fix(doc_vlm): remove ROCm BF16 _keep_in_fp32_modules workaround in PaddleOCR-VL #5077 未覆盖),所以两个 PR 任一先合后只需 trivial rebase 即可。建议合本 PR 时若 fix(doc_vlm): remove ROCm BF16 _keep_in_fp32_modules workaround in PaddleOCR-VL #5077 已先合,git 自动跳过_keep_in_fp32_modules的删除,余下runner.py部分继续生效。Test plan
paddlex.create_pipeline("PaddleOCR-VL")BF16 推理test_ocr.png端到端跑通。rocprofv3 --kernel-trace --stats:FP32 GEMM 调用从 18 756 → 1 316,GPU kernel 时间 4 415.7 ms → 3 915.5 ms(1.13×)。_keep_in_fp32_modules注释明确写其目的为 "ROCm stability (MIOpen bf16 conv has bugs)",移除后视觉塔仍按模型加载 dtype 运行;delete_pass块包在is_compiled_with_rocm()下,对 CUDA 路径无影响。